00100	
00200	             CRITICISMS OF PARRY
00300	
00400	                K.M. COLBY
00500	
00600	
00700		Much  of  the A-I community is now aware of our simulation of
00800	paranoid processes called PARRY.     The model has been available for
00900	interviewing  on  the  ARPA  network and thousands of interviews have
01000	been conducted with several versions of the model.  During  the  long
01100	period  of  development  of  the  model  we  have  been  aware of the
01200	potential limitations of various alternative  programming  approaches
01300	to  designing  an  algorithm capable of conducting useful non-trivial
01400	dialogue in natural language. Colleagues,  assocoates,  and  students
01500	have  volunteered  a  number of criticisms of the model.  I would now
01600	like  to  answer  them  en  bloc.    Only  serious,   reasoned,   and
01700	well-founded criticisms will be considered.
01800		Workers in A-I come from different  intellectual  traditions.
01900	This  influences  what people want from a model and would like to see
02000	in a model. Those from mathematical and logical backgrounds  like  to
02100	see  lots  of  deductive  inference; those from physics and chemistry
02200	like  to  see  laws;  those  from  the  life  sciences  like  to  see
02300	complexity,  growth, and development represented in a model.    It is
02400	important  that  we  recognize  and  respect   the   traditions   and
02500	philosophies of both the demonstrative and empirical sciences.  Those
02600	raised on a Euclidean model of knowledge seek to understand phenomena
02700	using  a  few  definitions and axioms, a few rules of inference, long
02800	chains of inference, and  deductive  consistency.   Some  aspects  of
02900	experience yield to this approach but many, especially in the case of
03000	biological organisms, do not.
03100		Everyone  realizes  that  a model represents a simplification
03200	and an idealization.  In constructing a model, only a  few  variables
03300	are  selected  as  centrally relevant while the rest are neglected as
03400	econdary or unknown.    Only a few  relations  between  the  relevant
03500	variables  are  introduced.  Thus a model does not match exactly that
03600	which it models in all details.  It is  partial  in  that  only  some
03700	aspects  of  the  referent  system  are  represented  and  it  is  an
03800	approximation in that it is limited in depth and not free of error. A
03900	model  is  an idealization in that it may utilize abstractions and it
04000	may possess perfect properties known to be  lacking  in  its  natural
04100	counterpart.   Thus the model's knowledge is not as extensive as that
04200	of a person  and  it  possesses  a  perfect  memory  unencumbered  by
04300	inhibitory  processes.  We  can  allow ourselves this idealization of
04400	perfect memory because we  are  not  studying,  for  example,  memory
04500	decay.  since  we  do  not  consider it to be a pertinent variable in
04600	paranoid processes.
04700		I  shall  list  the  major criticisms of the model which have
04800	come to our attention and attempt replies to each.
04900	
05000	CRITICISM #1:
05100		PARRY  is  simply  a  stimulus-response model.  It recognizes
05200	something  in  the  input  and  then  just  responds  to  it  without
05300	"thinking"  or  inferring.    The model should interpret what it sees
05400	and engage in more computation than execution of a simple rewrite  or
05500	production rule. REPLY:
05600		It is true that in early versions of the model  many  of  the
05700	responses  consisted  of  simple  rewrites,  e.g.     when  the input
05800	consisted of "Hello", the output response was "Hi" and no rules other
05900	than  of  the  type  "see x, say y" were involved. But as we began to
06000	improve and extend the model, this type of response disappeared.   In
06100	the  current  version,  the  model  consists  of two modules, one for
06200	recognition of natural language expressions  and  one  for  response.
06300	Once  the recognizer decides what is being said, the response module,
06400	using a number of tests and rules, decides how to respond.    Thus  a
06500	"Hello"  no  longer  receives  an  automatic  "Hi"  but may receive a
06600	variety of responses depending  on  a  large  number  of  conditions,
06700	including  a  "model" of the interviewer which PARRY builds up during
06800	the interview.   This  representation  of  the  interviewer  involves
06900	making inferences about his competence, his helpfulness, etc.
07000	
07100	CRITICISM #2:
07200		PARRY's performance constitutes  an  illusion.   The  model's
07300	data-base knowledge is too limited to represent adequately all that a
07400	person knows.   Because the model can answer a  few  questions  well,
07500	people,because  of  their  own  expectations,  are easily fooled into
07600	believing  PARRY  is  capable  of  answering  the  great  variety  of
07700	questions  a person is capable of answering. People will assume there
07800	is much more there than there really is.  Thus PARRY  is  a  fake,  a
07900	mirage,  a  conjurer's  trick in which the audience is led to believe
08000	something is true when it is not.
08100	
08200	REPLY:
08300		One  of Descartes' tests for distinguishing man from machines
08400	was that the latter "did not act from knowledge  but  only  from  the
08500	disposition  of  their  organs".   (Descartes'  other  test concerned
08600	linguistic variety). Granted that a model of a psychological  process
08700	should  contain  knowledge,  the questions become, how much knowledge
08800	and how is it to be represented?
08900		Since  a  model  is  a  simplification,   it   has   boundary
09000	conditions.    A  model  of  a  paranoid  patient is a model of being
09100	paranoid, being a patient, and being a person. PARRY does  reasonably
09200	well  in  the  first  two  of these "beings".   It fails in the third
09300	because of limited knowledge. How can we decide what the model should
09400	know?  It  is theoretically trivial to add tomes of facts to the data
09500	base but this seems to be what some A-I critics want. The  fact  that
09600	PARRY  can  discuss some topics rather well indicates it is doing the
09700	right things in these domains and could do well in other domains that
09800	are  functionally  similar. Simply adding facts without improving the
09900	algorithm can lead to a degradation of performance asexperience  with
10000	belief-system simulations and theorem-proving programs has shown.
10100		More important than sheer number of facts  is  how  they  are
10200	organized,  how they are represented, and how they are handled by the
10300	processing rules to contribute to the charactersitic  performance  of
10400	the  model.   Some seem happy to know there are fixed propositions or
10500	"frames" in  the  data-base  which  can  be  consulted  in  answering
10600	questions.  Even  if  a  model  can answer 50 questions about a topic
10700	using rewrite rules, some would say the model does not really  "know"
10800	anything about the topic. The procedural- declarative argument has no
10900	end in sight.  It  seems  to  be  a  matter  of  personal  style  and
11000	efficiency.
11100		PARRY is not a literal copy of a total person.  The  test  of
11200	adequacy  here  is not Turing's machine-question-"which is person and
11300	which is machine?" This is not a stringent test  since  the  criteria
11400	for  distinguishing  what  is human behavior over a teletype have not
11500	been systematically worked out, i.e. almost anything is  accepted  as
11600	being  human.    PARRY  is  not  the  real  thing;  it  is a model, a
11700	simulation,  an  imitation,  a  mind-like  artifact,  an   automaton,
11800	synthetic  and  artificial.     The  real  thing, a living person, is
11900	characterized by such  great  logical  complexity,  inhomogeneity  of
12000	class,  and individuality that a strategy of simplification is called
12100	for.
12200	CRITICISM #3:
12300		PARRY  models  paranoid  behavior   without   modelling   the
12400	underlying mechanisms of paranoid processes. Because the I-O behavior
12500	of PARRY is indistinguishable  from  the  I-O  behavior  of  paranoid
12600	patients, it does not mean that the same mechanisms are involved.
12700	
12800	REPLY:
12900		This is so true as to be an A-I truism.      When  the  inner
13000	mechanisms of a system are inaccessible to observation, one must make
13100	plausible guesses as to what is going on.   These  guesses  represent
13200	analogies.   They are not to be taken as the "same" mechanisms. If we
13300	knew the "  real"  mechanisms,  there  would  be  no  need  to  posit
13400	analogies about a hidden reality. We try to design structures to fill
13500	in the black box.    Further  empirical  tests  and  experiments  are
13600	necesary  to  increase  the  plausibility  of  the  analogy proposed.
13700	Successful  predictions  and  pragmatic   usefulness   increase   the
13800	acceptability  of  the  model  to  the  relevant  expert community or
13900	communities.
14000		We  can  never know with certainty whether a model is "true".
14100	If it is consistent with itself and with  the  data  of  observation,
14200	then  it is valuable cognitively and pragmatically. Such coherence is
14300	not a definition of truth but a criterion for truth.
14400		An expert community has various criteria for acceptability of
14500	a   model.   Sometimes  it  is  demanded  that  a  model  provide  an
14600	explanation.  What  constitutes  an  explanation   may   range   from
14700	describing  causes  to  making  intelligible  the connections between
14800	input and output. An extreme view is that science  does  not  explain
14900	anything;  A is simply interpreted in terms of B and B in terms of C,
15000	etc.
15100		A  pragmatic criterion for a model is whether it represents a
15200	a workable possibility?    Can it be tested and measureably  improved
15300	as a result of these tests? That is, is there an evaluation procedure
15400	for accumulative progress? In the case of PARRY, the answer to  these
15500	questions is "yes".
15600	
15700	CRITICISM #4:
15800	
15900		PARRY is an ad hoc model. It is designed after  the  fact  to
16000	fit a limited set of special cases and lacks generality.
16100	
16200	REPLY:
16300		Sometimes    this    criticism    is    levelled    at    the
16400	language-recognition  processes  and  sometimes  at  the scope of the
16500	model. The language recognizer of PARRY is a pattern-matcher. But the
16600	surface  English input expressions are transformed into more abstract
16700	patterns which are matched against stored patterns.  The  many-to-one
16800	tranformation involves synonymic-translations and word-classes.  Thus
16900	the language-recognizer has some generality in that  these  processes
17000	can be used by any "host" system which takes natural language input.
17100		It is true that PARRY is circumscribed.    It "explains"  the
17200	data  it  was designed to explain. One wants to achieve at least this
17300	degree of explanatory power in a model. But can it predict a new fact
17400	or  fit  a  new fact discovered in some other way?  This view sees ad
17500	hociness, not as a property of a model, but as a relation between two
17600	consecutive  models  or  theories.      Does  PARRY  have  some novel
17700	consequence compared to  its  predecessor?     One  trouble  is  that
17800	predecessor  formulations  explaining  paranoia  have been so vaguely
17900	stated as to be untestable.  The theory embodied  in  the  model  has
18000	novel consequences compared to other formulations.
18100		For example, the theory posits  that  the  paranoid  mode  of
18200	thought   involves  symbol-processing  strategies  which  attempt  to
18300	forestall or minimize the affect of humiliation.  A novel consequence
18400	of  this theory is that if a person were desensitized to the negative
18500	affect of  humiliation,  he  would  be  less  prone  to  utilize  the
18600	strategies of the paranoid mode.
18700	
18800	CRITICISM #5:
18900		PARRY'S paranoid behavior is strictly the  result  of  canned
19000	paranoid-like  responses. Granted that PARRY is diagnosed as paranoid
19100	by expert judges, this diagnosis is not a consequence of  the  theory
19200	embodied  in  the  model but is simply produced by the model's canned
19300	replies which are linguistically paranoid in nature.
19400	
19500	REPLY:
19600		
19700		This is a weighty  criticism  because  it  implies  that  the
19800	theory  of humiliation and the rules of the model are excess baggage.
19900	The  made-up  output  replies  are  so  typical  of  paranoid  verbal
20000	responses  that  they  alone might be sufficient to simulate paranoid
20100	interactions.
20200		Given that a model had a list of paranoid-like responses,  it
20300	would still need some mechanism or rules for selecting which response
20400	to output in reply to a specific input. Experiments have  shown  that
20500	random selection from this list results in an inadequate performance.
20600	For example, on a dimension of "thought disorder" on a 0-9 scale,  (0
20700	means  zero  amount  and  9  means  a  large  amount), a random model
20800	received a mean rating of 5.94 from  expert  psychiatrists.  Patients
20900	rated  by  the  same  judges received a mean rating of 2.99 whereas a
21000	version of PARRY was rated at 3.78.
21100		Little is known about how to generate surface  English  which
21200	is appropriate to the input  and phrased in a  characteristic  style.
21300	Segment-by-segment generation or even word-by-word  generation  would
21400	be  preferable  to  outputting  canned sentences as long as the rules
21500	posited for the paranoid mode were somehow called into  play  in  the
21600	generation  process.  (Fortunately  no  one  has  demanded that PARRY
21700	generate  words   letter-by-letter   to   account   for   alternative
21800	spellings).  Since  generation  of natural language output represents
21900	one of the major  shortcomings  of  the  model,  we  are  at  present
22000	attempting  to  couple  the  generation more closely with the model's
22100	theory.
22200	CRITICISM #6:
22300		The  model,  even if successful, is useless. Does it teach us
22400	anything about paranoia? Can it be used to  help  patients  suffering
22500	from paranoid disorders?
22600	
22700	REPLY:
22800		The model represents an attempt to make intelligible paranoid
22900	processes   in   explicit   symbol-processing  terms.    A  model  of
23000	psychopathology in which the mind is in error about some of  its  own
23100	processes  has  implications  for  prevention, reduction, and cure of
23200	disorder. PARRY  intersects  two  expert  communities  consisting  of
23300	researchers  in artificial intelligence and clinicians in psychiatry.
23400	Clinicians are practical men  who  are  interested  in  technological
23500	applications.
23600		If  the  disorder  is  at  the  "hardware"  level  of   brain
23700	pathology, then the application of symbol-processing techniques might
23800	be of little use.   But if there is reason to believe the disorder is
23900	at  the  program level of learned, acquired strategies, then attempts
24000	at re-programming  through  symbolic-semantic  techniques  are  worth
24100	considering.   At present clinicians have great difficulties treating
24200	paranoid disorders.  For a clinician practicing behavior therapy, the
24300	model's  theory  suggests desensitizing the patient to humiliation, a
24400	technique which has been successful with other negative affects  such
24500	as  anxiety.   For those practicing psychotherapy, the model's theory
24600	suggests exploring the topics of humiliation and self-censure in  the
24700	hope  of  helping  the patient to reject his judgements of himself as
24800	inadequate.  Judging whether these  treatments  are  effective  would
24900	depend on clinical evaluations.
25000	
25100	CRITICISM #7:
25200		PARRY does  not  tell  us  what  is  the  cause  of  paranoid
25300	thinking. Efective   treatment  requires  we  know  the  cause  of  a
25400	disorder.
25500	
25600	REPLY:
25700		PARRY  does  not account for how a system got to be that way;
25800	it describes only  how  the  system  now  works.  An  ontogenetic  or
25900	morphogenetic model would show how a normal system became that way as
26000	a result of its experience over time.
26100		It is not true that to have  effective  treatments  one  must
26200	know  the  cause  of  a disorder. Illnesses involve loops and circles
26300	which, if broken anywhere, can lead to relief of  the  disorder  even
26400	when  the  mechanism  of  action  of the treatment is not understood.
26500	Common examples of successful treatments  for  illnesses  of  unknown
26600	causes  are  insulin  in  diabetes,  digitalis  in  congestive  heart
26700	failure, colchicine in gout, and lithium in mania.
26800	
26900	CRITICISM #8
27000	
27100		The  tests PARRY has passed are not severe enough. If a model
27200	passes a validation test, it might not be because it is a good  model
27300	but because the test is weak.
27400	
27500	REPLY:
27600		Our strongest test invloves  having  judges  rate  interviews
27700	with  versions  of  the model and with paranoid patients.  We utilize
27800	statistical measures to  see  how  closely  the  model's  performance
27900	matches  that  of  the  patients and how much better it performs than
28000	previous model-versions. A recent study showed that on the  dimension
28100	of  linguistic  comprehension  independent  raters gave PARRY2 a mean
28200	rating of 5.48 on a  scale  of  0-9.  A  previous  version  of  PARRY
28300	received  a  mean  rating of 5.25. This improvement is significant at
28400	the 0.05% level. But the model is still  far  from  the  7.42  rating
28500	received  by  the  patients.   The  rating groups ( psychiatrists and
28600	graduate students ) have been shown to be  reliable,  i.e.  there  is
28700	agreement both within groups of raters and between groups.
28800		Stronger  tests  are  certainly  needed  and we would welcome
28900	suggestions along these lines. Are there validation tests others have
29000	used  which might be suitable for PARRY? In the past most models have
29100	relied on face validity. To improve  a  model  measureably,  we  need
29200	better tests and statistical measures.
29300	CRITICISM #9:
29400	
29500		PARRY is too sketchy, crude,  and  immature  a  model.   Such
29600	theoretical  models  can be premature for a field and can turn out to
29700	be irrelevant or counterproductive.   We  should  collect  more  data
29800	about    naturally-occurring   paranoia   before   attempting   model
29900	construction.
30000	
30100	REPLY:
30200	
30300		No  one really knows when to begin theorizing. Even facts are
30400	now believed to be  heavily  theory-laden,  whether  their  collector
30500	realizes  it  or  not.  One of the perils   of model building is that
30600	data used to test a model may demolish it. A model is only sufficient
30700	unto the day.
30800		If  PARRY  is  not  acceptable,  then  one accepts some rival
30900	formulation  (  a   current   one   is   "paranoia   represents   the
31000	transformation of love into hate"), or one accepts nothing and waits.
31100	Waiting for perfection can be paralyzing to a field, ,especially  one
31200	devoted to patients who need help.
31300		As  a  simplification,  PARRY  is  perhaps  too simple at the
31400	moment. In constructing a model, one strives  for  something  simpler
31500	than  the  "real" referent system which is difficult to understand or
31600	manipulate.  But  one  wants  to  retain   the   important   features
31700	characteristic  of  the  natural  counterpart.    If the model is too
31800	simple, it is  unable  to  reproduce  these  important  features  and
31900	extrapolation  to  the  natural referent system becomes risky. If the
32000	model is too complex, it  becomes  as  difficult  to  understand  and
32100	manipulate  as  the  real  thing.  Faced  with  this dilemma, a model
32200	builder can improve his model by simplifying it  or  making  it  more
32300	complicated while retaining consistency.
32400		These points are discussed in greater detail in a forthcoming
32500	book  entitled ARTIFICIAL PARANOIA: A Computer Simulation of Paranoid
32600	Processes to be published in a few months by Pergamon Press.